Commonly owned U.S. Pat. No. 7,123,277 entitled “CONVERSION OF A SUB-PIXEL FORMAT DATA TO ANOTHER SUB-PIXEL DATA FORMAT,” issued to Elliott et al., discloses a method of converting input image data specified in a first format of primary colors for display on a display panel substantially comprising a plurality of subpixels. The subpixels are arranged in a subpixel repeating group having a second format of primary colors that is different from the first format of the input image data. Note that in U.S. Pat. No. 7,123,277, subpixels are also referred to as “emitters.” U.S. Pat. No. 7,123,277 is hereby incorporated by reference herein for all that it teaches.
The term “primary color” refers to each of the colors that occur in the subpixel repeating group. When a subpixel repeating group is repeated across a display panel to form a device with the desired matrix resolution, the display panel is said to substantially comprise the subpixel repeating group. In this discussion, a display panel is described as “substantially” comprising a subpixel repeating group because it is understood that size and/or manufacturing factors or constraints of the display panel may result in panels in which the subpixel repeating group is incomplete at one or more of the panel edges. In addition, any display would “substantially” comprise a given subpixel repeating group when that display had a subpixel repeating group that was within a degree of symmetry, rotation and/or reflection, or any other insubstantial change, of one of the embodiments of a subpixel repeating group illustrated herein or in any one of the issued patents or patent application publications referenced below.
By way of example, the format of the color image data values that indicate an input image may be specified as a two-dimensional array of color values specified as a red (R). green (G) and blue (B) triplet of data values. Thus, each RGB triplet specifies a color at a pixel location in the input image. The display panel of display devices of the type described in U.S. Pat. No. 7,123,277 and in other commonly-owned patent application publications referenced below, substantially comprises a plurality of a subpixel repeating group that specifies a different, or second, format in which the input image data is to be displayed. In one embodiment, the subpixel repeating group is two-dimensional (2D); that is, the subpixel repeating group comprises subpixels in at least first, second and third primary colors that are arranged in at least two rows on the display panel. In some 2D subpixel repeating groups, the subpixels of two of the primary colors are arranged in what is referred to as a “checkerboard pattern.” That is, a second primary color subpixel follows a first primary color in a first row of the subpixel repeating group, and a first primary color subpixel follows a second primary color in a second row of the subpixel repeating group. Examples of such sub-pixel repeating groups are shown in FIG. 12.
Performing the operation of subpixel rendering the input image data produces a luminance value for each subpixel on the display panel such that the input image specified in the first format is displayed on the display panel comprising the second, different arrangement of primary colored subpixels in a manner that is aesthetically pleasing to a viewer of the image. As noted in U.S. Pat. No. 7,123,277, subpixel rendering operates by using the subpixels as independent pixels perceived by the luminance channel. This allows the subpixels to serve as sampled image reconstruction points as opposed to using the combined subpixels as part of a “true” (or whole) pixel. By using subpixel rendering, the spatial reconstruction of the input image is increased, and the display device is able to independently address, and provide a luminance value for, each subpixel on the display panel.
The subpixel rendering operation disclosed in U.S. Pat. No. 7,123,277 generally proceeds as follows. The input color image data from a portion, or area, of the input image is used to produce the luminance value for each subpixel on the display panel using an image filter comprising a matrix of coefficients. These coefficients are computed using a technique referred to as “area resampling.” The location of each primary color subpixel on the display panel approximates what is referred to as a reconstruction point (or resample point) used by the subpixel rendering operation to reconstruct a portion of an input image. Each reconstruction point is centered inside a resample area which defines the size of the area of the input image that potentially contributes to the luminance value of the subpixel. The set of subpixels on the display panel for each primary color is referred to as a primary color plane, and the plurality of resample areas for one of the primary colors comprises a resample area array for that color plane. The input color image data is represented as a set of tiled input image sample areas. The resample area array overlays the set of tiled input image sample areas such that each resample area overlays some portion of at least one, but typically more than one, input image sample area. The luminance value for the subpixel represented by a resample point is a function of the ratio of the area of each input image sample area that is overlapped by the resample area to the total area of the resample area.
The area resample function is represented as an image filter, with each filter kernel coefficient representing a multiplier for an input image data value of a respective input image sample area. More generally, these coefficients may also be viewed as a set of fractions for each resample area. In one embodiment, the denominators of the fractions may be construed as being a function of the resample area and the numerators as being the function of an area of each of the input sample areas that at least partially overlaps the resample area. The set of fractions thus collectively represent the image filter, which is typically stored as a matrix of coefficients. In one embodiment, the total of the coefficients is substantially equal to one. The data value for each input sample area is multiplied by its respective fraction and all products are added together to obtain a luminance value for the resample area (subpixel). The size of the matrix of coefficients that represent a filter kernel is typically related to the size and shape of the resample area for the reconstruction points and how many input image sample areas a given resample area overlaps.
In addition, in some embodiments of the techniques disclosed in U.S. Pat. No. 7,123,277, the subpixel rendering operation may be implemented in a manner that maintains the color balance among the subpixels on the display panel by ensuring that high spatial frequency information in the luminance component of the image to be rendered does not alias with the color subpixels to introduce color errors. An arrangement of the subpixels in a subpixel repeating group might be suitable for subpixel rendering if subpixel rendering image data upon such an arrangement may provide an increase in both spatial addressability, which may lower phase error, and in the Modulation Transfer Function (MTF) high spatial frequency resolution in both horizontal and vertical axes of the display.
Because the subpixel rendering operation renders information to the display panel at the individual subpixel level, the term “logical pixel” is introduced. A logical pixel may have an approximate Gaussian intensity distribution and may overlap other logical pixels to create a full image. Each logical pixel may be defined as a collection of nearby subpixels (e.g., at least one other subpixel) and has a target subpixel, which may be any one of the primary color subpixels, for which an image filter will be used to produce a luminance value. Thus, each subpixel on the display panel is actually used multiple times, once as a center, or target, of a logical pixel, and additional times as the edge or component of another logical pixel.
References to display systems or devices using more than three primary subpixel colors to form color images may also be referred to herein as “multi-primary” display systems. In a display panel having a subpixel repeating group that includes a white (W), or clear, subpixel, the white subpixel represents a primary color. Commonly-owned U.S. Patent Application Publication 2005/0225575, entitled “NOVEL SUBPIXEL LAYOUTS AND ARRANGEMENTS FOR HIGH BRIGHTNESS DISPLAYS,” discloses a plurality of multi-primary high brightness display panels and devices comprising subpixel repeating groups having at least one white subpixel and a plurality of saturated primary color subpixels. The saturated primary color subpixels may comprise red, blue, green, cyan or magenta in these various embodiments. Commonly-owned U.S. Patent Application Publication 2005/0225563, entitled “SUBPIXEL RENDERING FILTERS FOR HIGH BRIGHTNESS SUBPIXEL LAYOUTS,” discloses subpixel rendering techniques for rendering source (input) image data for display on display panels substantially comprising a subpixel repeating group having a white subpixel, including, for example, an RGBW subpixel repeating group. U.S. Patent Application Publications 2005/0225575 and 2005/0225563 are both incorporated by reference herein for all that each teaches.
FIG. 12 herein illustrates display panel 1570 substantially comprising an exemplary RGBW subpixel repeating group 9 which may be substantially repeated across display panel 1570 to form a high brightness display panel. RGBW subpixel repeating group 9 is comprised of eight subpixels disposed in two rows of four columns, and comprises two of red subpixels 2, green subpixels 4, blue subpixels 8 and white (or clear) subpixels 6. If subpixel repeating group 9 is considered to have four quadrants of two subpixels each, then the pair of red and green subpixels are disposed in opposing quadrants, analogous to a “checkerboard” pattern. Other primary colors are also contemplated, including cyan, emerald and magenta. US 2005/0225563 notes that these color names are only “substantially” the colors described as “red”, “green”, “blue”, “cyan”, and “white”. The exact color points may be adjusted to allow for a desired white point on the display when all of the subpixels are at their brightest state.
The subpixel rendering operation for rendering input image data that is specified in the RGB triplet format described above onto a display panel comprising an RGBW subpixel repeating group of the type shown in FIG. 12 generally follows the area resample principles disclosed and illustrated in U.S. Pat. No. 7,123,277, with some modifications and additions as described in US 2005/0225563. US 2005/0225563 discloses that input image data may be processed as follows: (1) Convert conventional RGB input image data (or data having one of the other common formats such as sRGB, YCbCr, or the like) to color data values in a color gamut defined by R, G, B and W, if needed. This conversion may also produce a separate Luminance (L) color plane or color channel. (2) Perform a subpixel rendering operation on each individual color plane. (3) Perform a sharpening operation using a sharpening filter. For example, use the “L” (or “Luminance”) plane to sharpen each color plane, or use a Difference of Gaussian (DOG) Wavelet filter to sharpen the image using a cross-color component or a self-color component.
In very general terms, a sharpening filter moves luminance energy from one area of an image to another. Examples of sharpening filters are provided in commonly-owned US 2005/0225563. A sharpening filter may be convolved with the input image sample points to produce a sharpening value that is added to the results of the area resample filter. If this operation is done with the same color plane, the operation is called self sharpening. In self-sharpening, the sharpening filter and the area resample filter may be summed together and then used on the input image sample points, which avoids the second convolution. If the sharpening operation is done with an opposing color plane, for example convolving the area resample filter with the red input data and convolving the sharpening filter with the green input data, this is called cross-color sharpening. In subpixel rendering operations in which a separate luminosity channel, L, is calculated, such as RGBW subpixel repeating groups, the sharpening filter may be convolved with this luminance signal; this type of sharpening is called cross luminance sharpening. These types of sharpening filters are typically constructed using a single primary color plane.
US 2005/0225563 discloses some general information regarding performing the subpixel rendering operation for RGB subpixel repeating groups that have red and green subpixels arranged in opposing quadrants, or on a “checkerboard.” The red and green color planes may use a Difference of Gaussian (DOG) Wavelet filter followed by an Area Resample filter. The Area Resample filter removes any spatial frequencies that will cause chromatic aliasing. The DOG wavelet filter is used to sharpen the image using a cross-color component. That is to say, the red color plane is used to sharpen the green subpixel image and the green color plane is used to sharpen the red subpixel image. US 2005/0225563 discloses an exemplary embodiment of these filters as follows:
TABLE 1−0.06250−0.062500.1250−0.06250.125−0.062500.250+0.1250.50.125=0.1250.750.125−0.06250−0.062500.1250−0.06250.125−0.0625DOG Wavelet Filter+Area Resample FilterCross-ColorSharpening Kernel
Commonly owned International Application PCT/US06/19657 entitled MULTIPRIMARY COLOR SUBPIXEL RENDERING WITH METAMERIC FILTERING discloses systems and methods of rendering input image data to multi-primary displays that utilize metamers to adjust the output color data values of the subpixels. International Application PCT/US06/19657 is published as WO International Patent Publication No. 2006/127555, which is hereby incorporated by reference herein. In a multi-primary display in which the subpixels have four or more non-coincident color primaries, there are often multiple combinations of values for the primaries that may give the same color value. That is to say, for a color with a given hue, saturation, and brightness, there may be more than one set of intensity values of the four or more primaries that may give the same color perception to a human viewer. Each such possible intensity value set is called a “metamer” for that color. Thus, a metamer on a display substantially comprising a particular multi-primary subpixel repeating group is a combination (or a set) of at least two groups of colored subpixels such that there exists signals that, when applied to each such group, yields a desired color that is perceived by the Human Vision System. Using metamers provides a degree of freedom for adjusting relative values of the colored primaries to achieve desired goal, such as improving image rendering accuracy or perception. The metamer filtering operation may be based upon input image content and may optimize subpixel data values according to many possible desired effects, thus improving the overall results of the subpixel rendering operation.
WO 2006/127555 also discloses a technique for generating a metamer sharpening filter which, in one embodiment, is a Difference of Gaussians (DOG) Wavelet filter. Metamer sharpening filters are constructed from the union of the resample points from at least two of the color planes. As explained in the commonly-owned WO 2006/127555 publication, the RGBW metamer filtering operation may tend to pre-sharpen, or peak, the high spatial frequency luminance signal, with respect to the subpixel layout upon which it is to be rendered, especially for the diagonally oriented frequencies. This pre-sharpening tends to occur before the area resample filter blurs the image as a consequence of filtering out chromatic image signal components which may alias with the color subpixel pattern. The area resample filter tends to attenuate diagonals more than horizontal and vertical signals. The metamer sharpening filter may operate from the same color plane as the area resample filter, from another color plane, or from the luminance data plane, to sharpen and maintain the horizontal and vertical spatial frequencies more than the diagonal frequencies. The operation of applying a metamer sharpening filter may be viewed as moving intensity values along same color subpixels in the diagonal directions while the metamer filtering operation moves intensity values across different color subpixels. The reader is also referred to WO 2006/127555 for further information.